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Abstract 

A profile of one state’s principal preparation programs is presented in this article. The collaborative 
inquiry process used to develop a program narrative instrument and to capture data on characteristics 
of all 17 state-approved programs in the state of Indiana is provided. The statewide preparation pro¬ 
gram characteristics profiled in this article include: missions and rationales, recruitment and admissions, 
student enrollment and completion, curriculum and pedagogical approaches, internship qualities, and 
assessment features. The author discusses implications of the findings across the preparation program 
landscape as well as the challenges and promises of collaborative research conducted across preparation 
programs. 
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1 Sumario en espanol 

Se presenta un perfil de programas de preparation de directores de escuelas en un estado estaduinidense 
en este articulo. Un proceso cooperative de investigation fue utilizado y los investigadores desarollaron un 
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instrumento narrative) para captar las caracterlsticas de todas las 17 programas de preparacion de directores 
de escuela en el estado de Indiana. Las caracterlsticas de los programas de la preparacion perfiladas en este 
artlculo incluyen: misiones, contratacion y admisiones, matriculation y graduation, enfoques pedagogicos, 
calidades de puestos de interne, y caracterlsticas de evaluation. El autor discute implicaciones de las con- 
clusiones a traves del paisaje de programas de preparacion de directores de escuela, incluyendo los desaflos 
y las promesas de investigation cooperativa realizada a traves de programas de preparacion. 

note: Esta es una traduction por computadora de la pagina web original. Se suministra como 
information general y no debe considerarse completa ni exacta. 


2 Introduction 

This article describes a collaborative inquiry process undertaken to capture information on program charac¬ 
teristics from across all 17 state-approved initial building-level administrative licensure (principal) prepara¬ 
tion programs in the state of Indiana. The program characteristics profiled in this article span the following 
features: missions and rationales, recruitment and admissions, student enrollment and completion, curricu¬ 
lum and pedagogical approaches, internship qualities, and assessment features. In the article, the author also 
describes how information on programs was captured through a collaborative inquiry process that included 
the development of a program narrative instrument and collective initial analysis of findings. Implications of 
the findings across the landscape of preparation programs are put forth, and a discussion of the challenges 
and promises of conducting collaborative inquiry across preparation programs concludes the article. 

3 Calls for Leadership Preparation Program Research and Evaluation 

Over the last two decades, university-based principal preparation has come under increased scrutiny by ex¬ 
ternal actors as a result of standards-based reforms, accreditation processes, and growing interest from public 
officials, private foundations, district administrators, and other stakeholders (Glasman, Cibulka, & Ashby, 
2002; LaMagdeleine, Maxcy, Pounder, & Reid, 2009; Murphy, 2006; Sanders & Simpson, 2005; Seashore 
Louis, Leithwood, Wahlstrom, & Anderson, 2010; Young & Brewer, 2008). There has been a shift in the 
preparation landscape as states have passed rules allowing various actors to prepare principals as new institu¬ 
tional providers and alternative certification policies have emerged (Harrington & Wills, 2005; Elmore, 2006; 
LeTendre, Barbour, & Miles, 2005; Smith, 2008). District and foundation supported alternative preparation 
programs and routes to certification have entered the preparation landscape, making the comparison with 
university-based programs inevitable (Barbour, 2005; Militello, Gajda, & Bowers, 2009; Teitel, 2006). Con¬ 
currently, the number of university-based educational leadership programs has risen, with new institutional 
actors becoming large producers of pre-service principals (Baker, Orr, & Young, 2007). 

Concern for the development of school leaders capable of leading reform and increasing student learning 
outcomes for all students is evidenced in state-level educational policy concerns and deliberations (Adams & 
Copeland, 2007; Sanders & Kearney, 2008; Southern Regional Educational Board, 2007), Wallace Foundation 
funded multi-state initiatives and studies (Frye, O’Neill, & Bottoms, 2006; Seashore Louis, et ah, 2010; 
Wallace Foundation, 2005), as well as University Council For Educational Administration (UCEA) and 
National Council of Professors of Educational Administration (NCPEA) reflective commentary, ongoing 
self-critique, and expanding interest in measuring program outcomes (Cambron-McCabe & McCarthy, 1999; 
Creighton, Harris, & Coleman, 2009; Glasman, Cibulka, & Ashby, 2002; Gonzalez, Glasman, & Glasman, 
2002; Hallinger, 2006; Kochan & Locke, 2009; Murphy, J. T., 2006; Orr, 2006; Pounder, 2004; Pounder, 
Reitzug, & Young, 2002; Young, Murphy, Crow, & Ogawa 2009). 

While these discussions and research efforts around evaluation and tracking of program outcomes devel¬ 
oped, other actors external to university-based educational leadership programs, who nevertheless have sub¬ 
stantial interest in the success of leadership preparation efforts, have sustained robust critiques of university- 
based leadership preparation programs and provided external pressure for reform (Frye, O’Neill, & Bottoms, 
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2006; Hess & Kelly, 2005). These critical voices portray the state of affairs in educational administrator prepa¬ 
ration as mediocre and inconsistent, and at worst, as “a race to the bottom” (Levine, 2005). While some 
within educational leadership have highlighted a history of innovative programs and approaches (Hull, 2003; 
Jackson & Kelley, 2005; LaPointe & Davis, 2006), others within the field also argue that little has changed 
despite these calls and little may change without significant external pressure and internal recognition of 
significant shortcomings in the way we prepare our future educational leaders (Elmore, 2005; Thompson, 
1999; Usdan, 2002). One significant way to engage the multiple permutations of these debates is through 
research on preparation programs, yet several educational leadership professors have pointed to a lack of 
systemic program evaluation work that systematically builds on previous research (Glasman, Cibulka, & 
Ashby, 2002; Halinger, 2006). 

4 Developing a Sustained and Systematic Self-study Tradition 

Despite such external pressures and concerns, sustained efforts aimed at gathering information on character¬ 
istics of principal preparation programs have often suffered from insufficient resources, infrastructure, and 
commitment, as well as competition across programs (Kochan & Locke, 2009; Orr & Pounder, 2006; Preis, 
Grogan, Sherman, & Beatty, 2007). Additionally, program inquiry has tended to focus on teacher education 
programs, aligning with the dominant focus of college of educations’ accreditation reviews (Monk, 2008). 

For many years efforts to investigate characteristics of principal preparation programs had been program 
specific case studies as universities devoted few internal university resources toward evaluating the efficacy 
of principal preparation programs (Murphy, 1992; Orr & Barber, 2009). As a result, more prevalent are 
case studies of programs (Heck & Hallinger, 2005) that often focus on program delivery mechanisms such 
as cohorts (Barnett & Muse, 2003), as well as research on specific university-district partnerships (Browne- 
Ferrigno, 2004). Program outcome measures have largely been captured through surveys of graduates that 
are program specific and tend to focus on student satisfaction and employment outcomes (Cambron-McCabe, 
1999; Kochan, Jackson, & Duke, 2005; Kochan & Locke, 2009). Too often, information from much of this 
activity tended not to be distributed widely and not even circulate amongst members of a faculty (Murphy 
& Vriesenga, 2004; Orr & Barber, 2009). 

Many within the educational leadership professoriate are responding to these challenges and other shifts in 
the preparation landscape with renewed interest in improving educational leadership preparation programs. 
This is reflected in extended commentaries and program evaluation efforts undertaken at the national and 
state level through NCPEA and its state-level affiliates, as well as LTCEA (Creighton, Harris, & Coleman, 
2009; Murphy, 2006; Orr & Pounder, 2006; Rorrer, Orr, Opfer, Scribner, Baker, & Perlich, 2010). More recent 
efforts have begun to make tentative links between programs and leadership behaviors and school outcomes 
(Darling Hammond, LaPointe, Meyerson, Orr, & Cohen, 2007; Hallinger & Heck, 1996; Seashore Louis 
et ah, 2010). In an effort to systematize and nationalize the collection of data on educational leadership 
preparation programs, the UCEA/LTEL SIG Taskforce to Evaluate Educational Leadership Preparation 
Program Effectiveness in cooperation with the University of Utah, has recently developed capacity to support 
the dissemination of the Taskforce graduate outcomes survey and a program narrative survey (based partially 
on the survey developed for this study), which should continue to better develop a profile of the field (Orr 
& Barber, 2009; Rorrer, Orr, Opfer, Scribner, Baker, & Perlich, 2010). 

4.1 Collaborative Approaches to Educational Leadership Preparation Research 

Historically collaborative work on preparation program characteristics or outcomes has sporadically occurred 
across programs in order to meet specific program needs (Clark & Clark, 1997) or professionalization efforts 
(Hull, 2003; Murphy, 2002). However, it is not surprising to note that collaborative inquiry that seeks to 
comprehensively describe and “map” the state of educational leadership preparation in individual states have 
been limited (LaMadgeleine, et ah, 2009). Efforts have been undertaken in Utah (Pounder & Hafner, 2006), 
Missouri (McCarther, Friend, & Watson, 2010), Illinois (Illinois Board of Higher Education, 2006), and 
New Jersey (Doolittle, Goduto, Monahan, Leake, & Murphy, 2006) to collaboratively capture characteristics 
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and/or outcomes from programs throughout the state. While individual research efforts may yield valu¬ 
able information on preparation programs, landscape and outcomes studies that require cross-program data 
collection on program characteristics, pedagogical processes, and selected outcomes necessarily require col¬ 
laborative efforts. Sustained and potentially efficacious translation and formative implementation of research 
at a regional or state level scale beyond an individual program requires more collaborative and politically 
sophisticated approaches (Orr & Barber, 2009; Young & Brewer, 2008). 

University-based faculty have conceptualized more collaborative approaches that variously seek to capture 
program characteristics and pedagogical processes, and subsequently trace the effects program characteris¬ 
tics and pedagogical processes may have on leadership behavior, and indirectly, various student outcomes 
(Ballenger, Alford, McCune, & McCune, 2009; Gonzalez, Glasman, & Glasman, 2002; Orr & Pounder, 2006, 
Newman, 2008). The UCEA/AERA Taskforce on Evaluating Leadership Preparation Programs Taskforce 
has been leading efforts to support collaborative program evaluation within the profession over the last sev¬ 
eral years. State-level program characteristic mapping efforts, such as the ones undertaken by this author 
and colleagues seek to first build relationships amongst program representatives and key state-level personnel 
in order to collaboratively document the delivery of program features (McCarther, Friend, & Watson, 2010). 
While the collaborative program evaluation work may be “politically challenging, [it] enables programs to 
benchmark their program delivery attributes and graduate outcomes, and provides much needed information 
on the impact of programs on graduates and the schools they serve” (Orr & Pounder, 2006, p. 7). Statewide 
program characteristic profiles allow program representatives, state officials and other stakeholders to benefit 
from research that can later evaluate the successive impact and relationship between leadership learning, 
career advancement, leadership practices, school improvement practices, changes in teaching and school 
organization, and changes in school outcomes. 

5 Methods 

In order to introduce the methods, a brief description of the context for the study is provided as a bridge to 
discuss the design of the program narrative inquiry instrument. Subsequently, collaborative data collection 
and analysis procedures are included in this section. 

5.1 Organizing for Collaborative Data Collection and Analysis 

From 2005-2006, a University of Indiana based team received funding from the Indiana Department of Ed¬ 
ucation’s Center for School Improvement and Performance to conduct a 14-month study of all 17 state 
approved building level (principal) leadership preparation programs, which resulted in a report published in 
2007 (Black, Bat lion. & Poindexter, 2007). Four years prior to the initiation of the study, the State of In¬ 
diana invested in educational leadership development through the adoption of ISSLC-based Building-Level 
Administrator Standards that guided a multilevel licensing process and a required program accreditation 
review for each of the then 11 accredited programs (which grew to 17 by 2005). The Indiana Department 
of Education had already established a multi-tiered licensure process that included a new administrator 
mentoring program, ongoing professional development for established administrators through the Indiana 
Professional Leadership Academy, as well as support for pre-service administrator development through the 
Indiana Promise Consortium, a group of select university preparation programs charged with innovating prin¬ 
cipal preparation (Balch, 2002). Indiana was also one of 15 states funded through the Wallace Foundation’s 
State Action for Educational Leadership Project (SAELP) initiative that aimed to retrain current leadership, 
recruit effective new candidates, and improve the practicing conditions of principals and superintendents in 
one urban school district (Orr, King, & LaPointe, 2010). 

The Indiana Building-Level Leadership Preparation Study was initiated with four objectives in mind: 

a. to comprehensively describe the state of educational leadership preparation in the State of Indiana. 

b. to report on national level efforts and methods utilized to evaluate and improve educational leadership 
preparation, c. To provide data that to inform policy decisions made at the state level as to how programs are 
approved/accredited to offer licensure and master’s degree programs in Indiana d. To provide data to colleges 
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and universities providing licensure and master’s degree programs in Indiana that would inform their program 
development and operational procedures. Study funding provided support for one principal Investigator, one 
graduate assistant, two consultants, and some small consulting fees and remuneration for professors from 
multiple institutions in Indiana to participate in data analysis. The study and the results reported in this 
article only covered licensure-only and masters plus licensure programs that lead to individuals’ obtaining 
their initial Indiana building-level administrator license (Black, Bathon, & Poindexter, 2007). 

Initial support for the study came to the State of Indiana through the Wallace Foundation, which has a 
strong interest in improving leadership preparation and funding research that highlights effective preparation 
program approaches. In this context, there was a push on the part of select parties within the state to not only 
conduct a descriptive “mapping” study, but to evaluate individual preparation programs. Through multiple 
conversations the research team communicated some of the technical and political difficulties involved in 
pursuing an evaluation of program efficacy. Beyond the limitations of a lack of clearly valid and reliable 
measures of program efficacy, focusing primarily on evaluation, the research team believed, would have been 
problematic in terms of the type of relationship building and trust that was needed not only to conduct the 
study and collect the data, but to engender future collaborative work. Actors within the state were divided 
as to the regulatory role of the state. Whereas some desired a more active role, others expressed that market 
forces, as reflected in enrollment numbers, would be the best arbiter of program efficacy. Additionally, many 
argued that the state’s involvement would become more bureaucratic than meaningful and substantive, 
representing additional cost in attending to compliance demands. The research team negotiated the multiple 
interests of both state and program level stakeholders by explicitly highlighting the mapping design of the 
study. Ultimately, funders at the state level settled for a study primarily investigating the “state of the state” 
of educational leadership preparation. 

5.2 Study Design and Instrumentation 

While the guiding framework was established, other design and instrumentation decisions had yet to be 
made. In the fall and winter of 2005-2006, the study team and an Indiana Department of Education 
official held two meetings with individuals representing a total of 14 of the 17 accredited Indiana principal 
program representatives in order to get input into a design of survey questions. A full day work session 
with the project consultants helped to organize the topical coverage and structure of the program narrative. 
Program chairs and other program representatives then were provided a month to review a draft of the 
program narrative inquiry instrument and provide feedback, which was later incorporated and reviewed by 
the external consultants and incorporated into the final narrative instrument. 

The program narrative was the central research instrument used to capture program-level descriptive 
information from all (n=17) accredited principal preparation institutions in the state of Indiana. The contents 
of the narrative inquiry instrument were generated through a review of literature on principal preparation 
programs, as well as in collaboration with both national consultants and peer representatives from other 
programs in Indiana. General content and structure of the instrument drew from the narrative developed 
by Pounder & Hafner (2006) in their study of programs in Utah. 

The 13 distinct program features or topical areas that covered in the program inquiry narrative in¬ 
strument included: Rationale, Leadership Standards, Program Structural Elements, Candidate Admission, 
Candidate Assessment, Program Curriculum and Curriculum Sequence, Teaching Methods and Pedagogical 
Approaches, Program Evaluation and Continuing Assessment, Program Field Experiences, Program Recruit¬ 
ment Strategies, Program Faculty, Program Strengths and Limitations and Distinctive Program Elements. 
The program narrative inquiry asked respondents from each preparation program to explain their program’s 
activities with regard to each of the specific topics through narrative responses. Within each of these topical 
areas, sub-questions guided the responses from each of the preparation programs. Some questions were 
designed to be quite narrow, while many others were open-ended. 

To generate depth and validity across program responses, specific information was additionally requested 
with respect to each topical area and evidentiary documents were requested to supplement the responses, 
when applicable and not overly onerous. This effort to capture complementary evidence was an attempt to 


http://cnx.org/content/m37384/1.2/ 



Connexions module: m37384 


6 


ensure greater reliability in program responses and to provide the researchers additional evidence. Examples 
of evidence requested included: mission statements, program syllabi, faculty vita, and internship handbooks. 
While the instrument reflected a survey with forced choices in some questions, the instrument was designed 
to collect narrative responses from programs. We also asked program respondents to attach specifically 
designated evidence. 

5.3 Data Collection 

Given the cooperative input on the design of the program narrative instrument, the majority of the program 
representatives were well aware of the study by the time we requested information. Importantly, when we 
sent out the survey to both program chairs and College of Education Deans, Dr. Sue Ellen Reed, Indiana 
Superintendent of Instruction, sent an accompanying letter asking program representatives to respond within 
two months. This high-level support was important in securing a high response rate. For the small amount 
of programs that did not initially respond, we followed up with phone calls and e-mails. As a result of 
significant investment of time, program narratives were submitted by each of the 17 approved programs 
we were able to collect over 1500 pages of documents. Additionally, recognizing the limitations of self- 
reported data, the researchers engaged in document analysis of program accreditation submissions to the 
state, analysis of program webpages, as well as conducted member checking through follow-up e-mails and 
numerous conversations with representatives of the different building-level leadership programs. 

The researchers frequently asked questions that the preparation programs simply did not know the 
answers to. Information on students had not been tracked over time and information on faculty had not 
been aggregated. Other information was not readily available either such as the courses offered in the past 
couple years, syllabi for all courses, assessment information, and full faculty vitae, to name a few. For the 
most part, leaders within the preparation programs were willing to exert the necessary effort to meet these 
self-reflection requests, but some questions were answered with N/As or simply no response was provided. 
Responses varied in depth and quality, as the approach is, to a large degree, dependent on the time and 
effort program representatives put into their responses. Thus, preparation programs had to engage in the 
process of gathering their own information to complete the survey. While all 17 programs responded, certain 
categories of characteristics had little information. 

5.4 Collaborative Analysis of Data 

After the program narrative information was received, an invitation to participate in analysis was extended to 
all leadership preparation programs. Eight program representatives accepted the opportunity. The research 
team invited program representatives to participate in analysis for multiple reasons. We had concerns related 
to our position and bias as employees of one of the 17 institutions. We also had a desire for the study to 
catalyze future collaboration and use of information across programs. In order to make this happen, we 
stripped program-identifying information from the data captured through the program narrative and shared 
with the participating analysts, which took an extensive amount of time. Nevertheless, we believe that this 
step did contribute to the willingness on the part of leadership preparation programs to submit information 
for analysis, as well as tempered preconceived biases any of our collaborating analysts might have brought 
to the table. 

Through the use of grant funds, we covered costs of housing and transportation to Indianapolis as well as 
compensation in the form of a small consulting fee to each of the eight analysts. The representatives came 
from large and small preparation institutions, as well as public and private preparation institutions. During 
the two-day discussion and training session participating analysts each received a full set of responses and 
affiliated evidence on one or two of the thirteen narrative sections. The participants were provided guiding 
questions for analysis that were reviewed and refined collectively as a group. Then, the analysts participated 
in a practice analysis session involving one or two topical sections of the narrative in order to establish 
norms and guidelines for analysis. This process offered professors from different institutions an opportunity 
to come together and provide perspectives and insight into the data, and express common concerns about 
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the field. It also allowed them to garner individual benefits in terms of payment and service opportunities 
that could be submitted in annual reports. This careful structuring of benefits to program representatives 
was an important catalyst for more robust engagement from programs and their representatives. 

The participants’ analysis took about a month and primarily described trends in particular program 
features across all programs in the state of Indiana. The analysts clustered categories and themes within 
each program feature domain. This was a substantial endeavor, as the participant analysis was edited for 
consistency of voice as well as subjected to consistency checks with all other evidence obtained. In conducting 
ongoing cross-case analysis, we took care to identify and analyze not only the data pertaining to specific 
program areas or features but also to encounter larger thematic consistencies across the multiple building- 
level leadership preparation program features and characteristics. Therefore, analysis included constant 
comparison across program feature areas in order to more broadly cluster patterns, establish variations, and 
identify potentially systemic patterns across all seventeen programs in the state (Stake, 1995; Yin, 1994). 
The multiple levels of analysis generated higher reliability in the data and validity in reporting, as well as 
promoted greater legitimacy with various study stakeholders. 

In the final report findings are presented across all program characteristics captured in the program in¬ 
quiry narrative instrument (Black, Bathon, & Poindexter, 2007). This article reports across the following 
domains: missions and rationales, recruitment and admissions, student enrollment and completion, curricu¬ 
lum and pedagogical approaches, internship qualities, and assessment features. Findings not included from 
the report include information on faculty, which is reported elsewhere (Black & Bathon, under review). 
Information on leadership standards was not included in this article because responses were fairly uniform 
across all programs. The information captured under program structural elements and the program strengths 
and limitations categories was largely captured in other domains. 

The discussion of findings begins with a description of program graduate production trend data captured 
through a second set of analyses. State officials assisted us by cross referencing two separate state datasets 
on new building-level licensures and employment data, which enabled us to analyze trends of recent program 
completers who received building administrator licenses. Multiple dimensions of licensure and placement 
outcomes for all preparation program graduates from 2001-2005 are reported elsewhere (Bathon & Black, 
2011; Black & Bathon, 2010; Black, Bathon, & Poindexter, 2007). 

6 Findings 

6.1 Graduation Trends 

There was a gradual, yet significant increase in the total number of initial licensures granted, as the number 
of building-level administrative licenses granted rose from 368 in 2001 to 435 in 2005 (an 18.2% increase). 
Paralleling the growth in licensed individuals is the growth in the number of preparation programs. In 
Indiana in 2001 there were 10 preparation programs, while by 2005 17 were accredited. Yet, the total 
number of employed school administrators remained relatively constant. 

Indiana Wesleyan University, a small private college in north central Indiana, began their leadership 
preparation program in 2000 and thus had zero graduates in 2001. However, by 2005 they were the largest 
producer in the state with 97 of the 435 licenses. While one private school was producing large numbers of 
potential principals, regional public universities were also large producers. Ball State University and Indiana 
State University produced 318 and 224 initial licensures between 2001 and 2005, respectively. Other longer 
standing programs witnessed stable or slightly declining enrollments, while a handful of relatively new, small 
private providers had very few licensed graduates. 

6.2 Divergent Program Missions and Rationales 

Although programs consistently aligned curriculum to the Indiana leadership standards, program missions 
and rationales were divergent. A handful of smaller programs reported that they did not have a specific 
mission statement and submitted a school of education mission statement. The program-level mission state¬ 
ments varied from a focus on effective leadership, to Christian and faith based orientations, and inquiry 
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and multicultural emphases. Variations in program mission narratives are presented below in Table 1 and 
categorized as broadly traditional (focusing on practical and instrumental purposes of training and reflecting 
effectiveness discourses); faith-based (centering purpose to faith-based and Christian perspectives or value 
orientations); or inquiry, care, and multiculturalism (often using language of social justice). 


Selected Indiana Building-Level Leadership Missions 


Traditional Program Missions 

Faith-Based Program Missions 

Inquiry, Care, and Multicultural¬ 
ism 

To prepare professional educators 
who have the knowledge, skills, 
dispositions essential for becom¬ 
ing reflective professionals, mas¬ 
ter educators, and educational 
leaders. 

To become lifetime advocates 
for Catholic education as lead¬ 
ers who serve the Church’s most 
valuable asset: her children. 

To transform educational institu¬ 
tions into nurturing and effective 
organizations through the cre¬ 
ation and application of knowl¬ 
edge, skills, and attitudes devel¬ 
oped through the collaborative 
preparation of educational lead¬ 
ers. 

To prepare administrators for 
professional service and leader¬ 
ship. 

To prepare school leaders who 
understand a Christian perspec¬ 
tive of life and their profession, 
and who will model how the 
Christian faith can be an integral 
part of the role of a teacher in 
both public and private schools. 

To develop high quality, caring 
professionals who stimulate con¬ 
tinuous renewal of schools within 
a multicultural society. 

To prepare highly qualified school 
leaders who serve children by pro¬ 
viding exemplary leadership. 

To promote academic knowledge, 
technological skills, pedagogi¬ 
cal proficiency, life-long learning, 
Christian ethical and moral val¬ 
ues, enhancement of each candi¬ 
date’s intellectual, spiritual and 
social development, and com¬ 
munity service through positive 
leadership. 

To be the premier program in 
the preparation of working pro¬ 
fessionals for administrative lead¬ 
ership in elementary, secondary, 
and post-secondary education. 


Table 1 


Two programs referred to becoming the premier educational leadership preparation institution in the 
state, reflecting the competitive nature of school leader preparation. A few programs referred to caring or 
nurturing leadership training for a multicultural society. Another interesting finding is the large number of 
programs that did not specifically refer to preparing educational leaders as their primary mission. Many 
programs referred to preparing program professionals and some programs even referred to their mission 
of preparing teachers. This likely reflects the central position teacher preparation occupies in college of 
education. However, the program missions’ lack of specific focus on preparing educational “leaders” or 
“administrators” and use of more general language around educational practice was striking as many might 
assume that it is the central organizing principal of building-level leadership programs. Finally, one statewide 
trend that all programs seemed to agree upon is the use of the terminology of “educational leadership” instead 
of the terminology of “educational administration,” reflecting shifts in the language of the field reflected in 
the ISLLC and Indiana Building-Level Administrator standards. 
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6.3 Recruitment and Admissions 

Programs attempted to recruit potential students in three primary ways: word of mouth, brochures, and 
websites. Slightly less than 30% of Indiana building-level administrator programs (n=5) reported formal 
linkages with one or more school corporations that lead to the establishment of important recruitment 
networks. One program distributed brochures to teachers in schools throughout the state. Websites varied 
significantly in quality of information and ease of use for potential students. 

Across all 17 accredited programs candidates had to submit an undergraduate transcript with a 2.8 GPA 
mean as a minimum requirement. Virtually all programs (93%) solicited letters of recommendation. More 
than half (63%) required a goal statement from candidates, while an equal percentage (63%) of students were 
interviewed as part of the admissions process. While the GRE has historically been used as a central element 
of admissions decisions, less that half of the programs (43%) listed the GRE as an admission requirement. 
For those programs requiring the GRE, the statewide average minimum score was 837.5. Once students met 
minimum criteria, the programs did not appear to have selective admissions. Only three programs reported 
an admissions rate below 90 percent and five programs stated that they had a 100 percent admissions rate. 
The statewide average program acceptance rate was 93% of applicants. 

6.4 Student Enrollment and Completion 

Most building-level preparation programs (76%) offered both licensure and licensure accompanied with a 
Master’s degree. The number of credit hours required in licensure-only programs of study ranged from 
24-37, while the number of credit hours programs required for masters degrees ranged from 36-42 hours. 
The shortest identified time to program completion was 14 months for one program while the longest time 
reported for program completion stretched to 60 months for another program. 

The majority of programs arranged students in cohorts. Not only were more women enrolling in building- 
level leadership programs than men, but also of the persons who enrolled, more women completed the 
program. By the time students finished their programs, a clear majority of women were earning their initial 
building administrator license (55% women to 45% men). The statewide percentage of minority enrollees 
was around 8 percent. Collectively, programs did indicate that the minority candidate program completion 
rate was lower than the majority (white) completion rate. 

6.5 Program Curriculum 

Partially as a result of programs’ attempts to align building-level administrator preparation to the Indiana 
standards, six to eight core classes and a handful of unique supplemental courses were featured across the ma¬ 
jority of programs in the state. These courses included educational leadership, school law, the principalship, 
curriculum, school-community relations, and the internship or practicum. Fewer than half of the programs 
in the state of Indiana offered electives of any sort, and most programs had a predetermined course sequence 
program candidates were required to follow, which is consistent with cohort models. In slightly more than 
half of programs the course instructor developed the course syllabi without a predetermined syllabus tem¬ 
plate. An additional five programs encouraged course instructors to construct their own syllabus within 
the guidelines of a predetermined course template. Review of syllabi indicated that instructional leadership 
was the topical area most commonly addressed across distinct courses in a significant minority of classes, 
while issues of diversity and cultural competence were generally not addressed except in school-community 
relations courses. 

6.6 Pedagogical Approaches 

Methods of instruction did not appear to vary widely across programs in Indiana, although there was a 
significant variation in use of technology and distance learning both within and amongst programs. Instruc¬ 
tors most frequently utilized problem-based learning, case studies, and extended class discussion, as over 
half of the programs reported consistent use of these methods. The use of field-based approaches was not 
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often tied to a single course, as in many programs field-based activities ran throughout the program. While 
the majority of courses were still offered on university campuses, programs reported trends toward offering 
instruction off-site, as programs reported efforts to work with school districts and regional service centers. 
Half of the programs in Indiana and all of the larger programs offered some instruction in their building-level 
administrator programs in k-12 school or community settings, or via online delivery systems. 

6.7 Internship Qualities 

All programs (17) reported an internship or practicum as the primary field based experience. Two programs 
did not have a specific course for the primary field based experience, but rather embedded field-based expe¬ 
riences throughout the program. Six out of the seventeen programs indicated that a formalized relationship 
existed with nearby school districts that served to grant candidates field based administrative experience. 
Most often, course credit was given for a primary field-based experience scheduled near the end of a candi¬ 
date’s program, and most candidates arranged for their placement at a school or schools. As a result, the 
majority of the programs reported that most candidates were able to complete the field-based experience in 
their home schools while still working full time as teachers. 

At the time of the study, Indiana stood out for the low number of hours required: 11 out of 14 program 
respondents required 150 hours or less for their internship. Only three programs reported a contact hour 
requirement of 300 hours, while one reported 60 hours. There was a nearly even divide between the number 
of programs offering the internship as a three credit hour, one semester class and those offering the internship 
as more than a one semester, three credit class, typically a six credit course. The research team found that 
the evaluation of the internship differed greatly by program. Some programs had heavier documentation 
requirements with fully developed internship handbooks, significant use of journals or portfolios, and super¬ 
visors who were required to submit written logs or complete observation surveys on interns. Other programs 
reported that professors relied more heavily on oral communication with the student and the supervisor in 
the form of interviews and anecdotal evidence. 

6.8 Assessment Features 

The majority of programs structured their assessment of student progress as a three-stage process. The first 
assessment of a student occurred at the gateway process for admission, while the second stage tended to 
happen as a midpoint assessment tied to entrance or exit from the practicum. The final assessment point 
occurred at the termination of the building administrator program when students turned in a portfolio, 
demonstrated a minimum grade point average, and were expected to take and pass the School Leaders 
Licensure Assessment (SLLA). For the 10 programs that did report the SLLA as an assessment tool used by 
their program, there was an extremely high passage rate over the last two years, with the lowest program-level 
student passage rate for any one-year period being 96%. 

The majority of programs utilized class grades/assignments and portfolios as their primary means for 
candidate assessment at the program level. In most programs students were required to maintain a minimum 
grade point average on all coursework. The vast majority of programs utilized some form of a portfolio model. 
However, as reported by one program: “We call our portfolio a Standards Journal. We do have a rubric for 
the standards journal, although in practice the journal is more often an occasion for a conversation between 
the faculty member and the student, concerning how the student is progressing relative to the standards.” 
Mid-point student interviews were also another frequently reported assessment procedure used by programs. 
In some cases, this mid-point oral assessment was used as a gateway to the internship. 

Programs consistently reported that they were evaluated and made program modifications in response to 
state and NCATE program reviews. Nevertheless, how programs used the state’s Unit Assessment System 
(UAS) 2 and National Council for the Accreditation of Teacher Education (NCATE) evaluation and assess¬ 
ment processes forformative, ongoing self-evaluation appeared inconsistent. Programs that indicated that 

2 The UAS process is a Indiana Department of Education document submission and review process scheduled one year before 
the NCATE program review. It is designed to be formative, but also to assure that programs and colleges of education have 
systematic data collection protocols. 
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the UAS process was a program evaluation and improvement tool stated that their primary use of the UAS 
process was at the school of education level, and not at the departmental or building-level administrator 
program level. Six programs identified either an oversight committee or an oversight coordinator who reviews 
program data on an annual basis. The other program assessment and evaluation tools used by less than half 
of the programs were program exit and alumni surveys. In addition, annual reviews by higher institutional 
authorities, a student advisory council, employer surveys, student surveys, outside program evaluation com¬ 
mittees, and faculty retreats were reported as assessment mechanisms in Indiana. By 2006, forty-one percent 
of programs in Indiana actively survey their graduates upon completion. 

Programs may track program data for external evaluation and assessment purposes, yet program represen¬ 
tatives commented in their narrative responses that this information was not frequently or effectively shared 
with program faculty. Program representatives generally reported that data such as SLLA passage rate, en¬ 
rollment, retention, and completion data, and student employment and contact information data were rarely 
systematically shared with program faculty. Whereas only half of the programs reported tracking graduates, 
programs did report interest in establishing tracking methods that would assess the graduate/completers’ 
impact on student learning and school reform. 

7 Implications from the Program Characteristic Findings 

It was clear that the program mission statements guided curricular decisions and represented the diversity 
of private, faith-based, and public universities. Nevertheless, we did not find a huge difference across pro¬ 
grams as issues of finance, law, instructional leadership that reflect standards based approaches seemed to 
predominate. Further efforts to craft out a niche, and to align program activities to the mission might be 
warranted, particularly in a crowded principal preparation landscape. 

In terms of recruitment, many programs did attempted to work with districts. However, exceptional 
program indicators around collective selection of students (Jackson & Kelley, 2002; Orr, King, & LaPointe, 
2010) was not evidenced in the program responses. The incredibly high admissions rate also suggests more 
selective admission processes that align with more exceptional programs might be warranted (Darling Ham¬ 
mond, et ah, 2007). Clearly, the use of the GRE has also diminished and using it as a requirement seems to 
put some programs at a competitive disadvantage. 

Student enrollment trends suggested that programs that aggressively marked, use distance technology, 
and teach in schools settings that are convenient to students tended gain enrollment. One area in which 
research has consistently supported principal program features and practices, surrounds efficacy of cohort- 
based delivery approaches (Barnett & Muse, 2003). The programs in Indiana have seemed to embrace 
this research. Additionally, they have very high completion rates amongst students in cohort, albeit they 
are slightly lower for the small amount of students of color. Specific efforts designed to attract and retain 
students of color seems to be warranted (Jean-Marie, Normore, & Brooks, J., 2009). 

Program curriculum narrative responses demonstrated an extremely common use of course sequences with 
minimal amounts of electives, which is consistent with some cohort delivery models. Consistent with the 
ISLLC standards and commentary and research in the field (Darling-Hammond, et. ah, 2007), instructional 
leadership seemed to be the most central curricular element. Nationally more programs have also featured 
the development of instructional leadership capacity as an essential program orientation (LaPointe & Davis, 
2006). Interestingly, issues of diversity and cultural competency seemed to be concentrated in one class- 
the community relations class. With the exception of two programs, little was mentioned around students 
with disabilities and English Language Learners. In terms of pedagogical practices, many programs had 
experienced school leaders that would connect the curriculum to practice through case studies and field 
embedded experiences. 

The range in internship hours and structures was a surprise. Consistent with critiques of the field, most 
students completed their internships at their home schools while working ful time. Given the large amount of 
resources necessary to effectively plan and deliver a multi-site internship, this might be an area in which the 
state invests in students attending school full time, as it does in states like Mississippi and North Carolina 
(Darling-Hammond, et. ah, 2007; U.S. Department of Education, 2004). Regardless, the state of Indiana’s 
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recent move to the ELLC standards for the internship, which require a 300 hour internship, is a move toward 
a more coherent and robust leadership development system. 

While the SLLA may provide a measure of minimal “do no harm” competencies (Adams & Copeland, 
2007), given that almost all students who applied to a program were admitted, and 96% of students statewide 
passed the SLLA, programs and state officials should engage in a dialogue about systematic selection and 
assessment systems. Does there need to be more gateway assessments and how might they be structured? 
This could then provide entree to utilizing and even designing more research that seeks to link the character¬ 
istics of programs to leadership behaviors and indirectly, school reform. (Jackson & Kelley, 2002; Leithwood, 
Jantzi, Coffin, & Wilson, 1996; Murphy, 2006; Orr & Pounder, 2006). 

8 Implications for Collaborative Inquiry on Principal Preparation Programs 

8.1 Creating Conditions For Participation 

One of the more substantial challenges involved recruiting faculty members from each of the accredited 
programs to respond to initial surveys, participate in interviews, or participate in ongoing analysis and 
formative use of data. Beyond meeting accreditation requirements, most of the preparation program leaders 
were not engaged in national-level conversations on program self-evaluation. While many of the preparation 
programs were familiar with the external criticisms of the field, particularly the Levine (2005) report, few 
of the preparation program representatives were initially familiar with the field’s internal response to this 
growing pressure (Orr &Pounder, 2006). Yet many expressed confidence that despite potential challenges, 
inquiry across programs was a timely and important endeavor. 

Planning for time to explicitly discuss common interests was an important first step in this study’s design. 
The program narrative instrument was sent out for review to program representatives and feedback was 
incorporated into the narrative. The researchers encouraged collaborative analysis and shared preliminary 
findings with program representatives for a little under a year before a report was released. Thus, the 
programs were well aware not only of the information likely to be contained in the report, but also the 
personality and tenor of the research team. 

Identifying and negotiating distinct program self interest is also an important consideration for future 
research. As the number of new programs in the state rose and one in particular quickly enrolled large 
numbers of students, several established (pre-2000 era) program representatives were motivated to come 
together to document the newly shifted landscape in the face of perceived or real threats to losing their 
market share of potential enrollees. In the one new preparation programs that gained so many students 
over the period of 5 years, the traditional leadership structures of a tenured program chair and several 
supporting full time faculty was replaced with a preparation program structure that included only one 
or two fulltime employees serving as program directors that guided the development of modules delivered 
almost exclusively by adjunct instructors. Interestingly, during the study the representatives of this program 
did provide information, although they did not respond to opportunities for collective analysis, discussion, 
and presentation of data. Ultimately, during three meetings with an average of 9 out of the 17 program 
representatives present, several program representatives expressed desire to limit the growth of programs 
they perceived to lack rigor and to increase state level regulations all programs in order to limit adjunct 
heavy programs that had grown so rapidly. Multiple program representatives saw this model as a threat to 
their professional identity, and a more rigorous mode of study represented in their preparation programs. 
This sense of shared interest was further solidified when enrollment numbers were presented and shared that 
demonstrated that as the one program grew dramatically, other programs had slightly declining enrollment 
(Black & Bathon, 2010). 

How to accommodate the perceived threats and distinct orientations between programs was a politically 
tricky position to be in while conducting necessarily collaborative research. These types of tensions were also 
evidenced in the Missouri Professors of Educational Administration group deliberations, where discussion 
over the appropriateness of high use of adjuncts caused friction amongst different program representatives 
(McCarther, Friend, & Watson, 2010). Creating a space to openly share concerns and withhold judgment 
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as to program quality was an important leadership action for the research team to undertake and having a 
respected and well-known colleague on the team assisted this endeavor. For this initial inquiry, disidentifying 
data during the analysis process and reporting program characteristic data across programs rather than as 
program specific characteristics turned out to be valuable strategic decisions. 

8.2 State and Funding Agencies’ Roles 

Leaders in funding agencies and state department of education offices are crucial to helping preparation 
programs inquire into their own practices. These leaders can convene meetings, establish commitment from 
new preparation programs, identify internal leaders, form research agendas, and encourage responses from 
preparation programs to self-evaluative findings. In Indiana, the state department of education played a 
vital role in encouraging participation. The research team found it useful to send out the official request to 
the programs on state letterhead with the explicit support of the state officials. The collaboration with state 
officials with some measure of regulatory power, along with phone calls, e-mails, and personal relationships 
enticed all programs to submit their program narrative and supplementary documents. 

An ongoing challenge relates to time and funding. Statewide, self-evaluative research does have a cost, 
and sustaining the work requires faculty buy-outs, administrative support and aid to support graduate 
students. Without some support from either funding agencies or state entities it is unlikely that researchers 
will engage in the arduous task of statewide preparation program research as the academic benefits of such 
research are still relatively low compared to other types of scholarly activities (LaMontaigne, et al., 2009; 
Orr & Pounder, 2006). 

8.3 Sustaining Inquiry into Preparation Programs 

An issue brought forward by program representatives was the lack of funding for preparation program inquiry 
and the feeling amongst all faculty members that they were stretched in many different directions. This has 
been an issue for quite some time, although external funding and national networks have become more 
common. Calls have been clear in articulating the need for cross-program evaluation supports and internal 
program resources dedicated to using program data to inform planning and to sustain the political legitimacy 
of the field (Orr & Barber, 2009; Young & Peterson, 2002). 

Program-level leadership that committed to preparation program inquiry was exceedingly important. Ad¬ 
ditionally, intrastate leadership was needed to continue the conversation across programs. The research team 
had a current university professor who had extensive contacts across school districts, the state department 
of education, as well as personally knew several program representatives well through her superintendent 
placement consulting work, which proved to be an extremely valuable asset for overcoming initial concerns 
and logical resistance to the additional work of collection and sharing of program-level information. 

Changes in statewide leadership at university programs and the state departments of education, and the 
drying out of external funding have proven to be substantial issues. When the primary researchers on this 
study left the state, the momentum toward collaborative inquiry was not sustained at the program level. 
What became clear is that formal associations that garner support of state departments of education are 
one way to maintain momentum. Yet to address sustainability, collaborations must be creative in providing 
reasons for continued collaboration. Statewide journals or conferences for the publication of state-specific 
program evaluation work, for instance, could be a good way to foster collaboration and provide value to 
academically oriented faculty while at the same time providing a service to the state that could provide 
information to school districts and the state departments of education. Collectively designed longitudinal 
studies that focusing on transfer of program knowledge into student learning and practice could have outlets 
in these conferences and journals and involve state licensure personnel and other stakeholders (Baker & 
Cooper, 2005, Barnett, 2005; Newman, 2008). Collaborative blogs or wikis on state specific educational 
leadership issues may be another outlet for creative collaboration. Program preparation collaborations may 
assist in the sustainability of efforts in ways that meet individuals’ professional obligation for service and 
research that can now be published in UCEA and NCPEA sponsored journals that are receptive to leadership 
preparation program studies. 
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9 Conclusion 

The effort in Indiana suggests that if representatives from distinct preparation programs have the opportunity 
to view their efforts as primarily collaborative rather than competitive, preparation programs are much more 
apt to learn from each other strengths. For example, there was much conversation about the internship 
qualities and ways in which field experiences were embedded through coursework. While the aspect of 
competition never fully goes away, the majority of preparation programs can remain collaborating because 
they understand that schools and children are better off when preparation programs across the state are 
pulling on the rope in the same general direction (McCarther, Friend, & Watson, 2010). 

While additional and methodologically distinct investigations are necessary, particularly studies that 
track completer learning, behavior, and school reform efficacy and impact on multiple student outcomes 
(Orr & Barber, 2009; Seashore Louis et ah, 2010), this type of study represents a first step that is likely 
to encourage more understanding on the part of the state actors and action on the preparation program 
representatives themselves. Proactively defending educational leadership preparation from outside attack, 
strategically responding to fluid market pressures, and engaging in self-improvement is an ambitious affair, 
but there may be no better first step than examining our own practices. 
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